Ontologies for Reasoning about Failures in AI Systems

Brittleness is a common problem among AI systems. Autonomous systems, including those that learn, may be faced with unanticipated situations that cause decreased performance, or in the worstcase, catastrophic failures from which the system cannot recover. In this paper, we describe a construct called the metacognitive loop (MCL) that allows AI systems to monitor their own behavior, generate expectations about their own progress and performance, and verify that they are met. When expectations are violated, the metacognitive loop attempts to reason in a domain-general way about why expectations were not met and how to recover. The basis for reasoning is a set of ontologies that encode abstract diagnosic and prescriptive processes for coping with failures.diagnosic and prescriptive processes for coping with failures.